fix(vt,hooks): async readback is a Backend capability + unbreak the pre-push sandbox - #159
Merged
Conversation
…re-push sandbox (#159) Two guards on main were RED and between them blocked every open external contributor PR (#127, #154, #155) and every push. Both premises were verified in the tree before changing anything. 1) check-device-leakage: src/vllm/v1/worker/gpu/runner.cpp named vt::DeviceType::kCUDA in the device-agnostic shared layer (DSR bucket 'kcuda' 1 > baseline 0). It came in with the QueueSupportsAsyncInputCombine rescope during the PR #140 fix round - ours, not the contributors'; richiejp reported it in #127's honest gaps. Fixed the way the guard's own message prescribes, mirroring the SupportsAuxStream precedent: ask the backend, not the device. New vt::Backend::SupportsAsyncSampledTokenReadback() (base false) answers whether the host may validly read the sampled token id back between steps; CPU overrides true (host and device memory are one allocation) and CUDA overrides true (the id is device-mirrored). The runner asks vt::TryGetBackend(queue.device.type), whose nullptr for a device absent from the build also subsumes the old #ifdef VLLM_CPP_CUDA guard. SEMANTICS UNCHANGED: CPU async-ON, CUDA async-ON, discrete non-CUDA (ROCm gfx1201) async-OFF - the "!"-token hazard stays closed. 2) .githooks/pre-push ran check-policy.py inside a PARTIAL export (README.md docs scripts .agents), but policy_contract.py:428 asserts AGENTS.md is a non-symlink regular file and resolves its Markdown links against that sandbox. AGENTS.md and its .env.example link were both missing, so the hook failed closed on content that is fine in the real tree - every push refused. EXPORT_PATHS is now a superset of what the CHECKERS read. Gates: check-device-leakage RED->GREEN (kcuda=0, DSR 32 == baseline 32); all four hook checkers OK in the reproduced sandbox; test_async_llm 8/8-347, test_engine_core 6/6-44, test_llm_engine 11/11-204 (CPU still resolves async-ON); clean -Werror CPU build; full 11-gate record battery green. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude (Opus 5) via Claude Code
mudler
force-pushed
the
row/FIX-DSR-ASYNC-READBACK
branch
from
August 8, 2026 19:54
6ebf48e to
1f3b714
Compare
This was referenced Aug 8, 2026
mudler
added a commit
that referenced
this pull request
Aug 8, 2026
perf(gdn): dispatch exact causal-conv chunks Contributed by Richard Palethorpe <io@richiejp.com>; merged with --no-ff so the four authored commits land under their own authorship. Mirrors upstream compute_causal_conv1d_metadata: grid.y becomes a flattened list of (sequence, 8-token chunk) programs instead of a rectangular grid that either padded or ran sequences serially. The descriptor is optional on CausalConv1dArgs and validated as a pair in vt::CausalConv1dFwd; CUDA consumes it, CPU keeps its scalar reference. Default ON per the parity-enabler policy, with VT_CONV_EXACT_CHUNKS=0 as the same-binary rollback. Contributor measured 720.047 -> 234.607 ms (3.07x) on the causal-conv family, sm_120 Qwen3.5-4B c32, byte-identical output tokens. The branch's only red check was device-leakage, which was OUR blocker: runner.cpp named DeviceType::kCUDA in the device-agnostic layer. Fixed on main in #159. Verified on the merge result: every scripts/check-*.py green, clean CPU build 1103/1103, ctest 356/356 (test_engine_core_proc fails only under -j8 and passes serially, the known parallel-starvation flake). CI on the #165 landing branch also had build-test-cpu, build-test-vulkan and device-leakage green. Conflicts, three files, all records: .agents/state.md and .agents/benchmark-record.md unioned per the append-log rule; scripts/merged-gemm-consistency-allowlist.txt had a gemma4_moe key added independently on both sides, kept as one entry with main's text. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Opus 5 (1M context)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two guards on
mainwere RED and between them blocked every open external-contributor PR (#127, #154, #155) and every push. Both premises were verified in the tree before changing anything.1.
check-device-leakage— the shared layer named a devicesrc/vllm/v1/worker/gpu/runner.cppnamedvt::DeviceType::kCUDA(DSR bucketkcuda1 > baseline 0). It came in with theQueueSupportsAsyncInputCombinerescope during the PR #140 fix round — ours, not the contributors'. @richiejp reported it in #127's honest gaps and was right.Fixed the way the guard's own message prescribes, mirroring the
SupportsAuxStreamprecedent: ask the backend, not the device. Newvt::Backend::SupportsAsyncSampledTokenReadback()(base false) answers whether the host may validly read the sampled token id back between steps — CPU overrides true (host and device memory are one allocation), CUDA overrides true (the id is device-mirrored). The runner asksvt::TryGetBackend(...), whose nullptr for a device absent from the build also subsumes the old#ifdef VLLM_CPP_CUDA.Semantics unchanged: CPU async-ON, CUDA async-ON, discrete non-CUDA (ROCm gfx1201) async-OFF — the
"!"-token hazard stays closed, and the ROCm TODO now points at an override instead of a device branch.2.
.githooks/pre-pushfailed closed on fine contentThe hook runs
check-policy.pyinside a partial export (README.md docs scripts .agents), butpolicy_contract.py:428assertsAGENTS.mdis a non-symlink regular file and resolves its Markdown links against that sandbox.AGENTS.mdand its.env.examplelink were both missing, so every push was refused with errors that do not reproduce in the real tree.EXPORT_PATHSis now a superset of what the CHECKERS read.Gates
check-device-leakageRED → GREEN (kcuda=0, DSR 32 == baseline 32)test_async_llm8/8·347,test_engine_core6/6·44,test_llm_engine11/11·204 — CPU still resolves async-ON (the regression this could have caused)-WerrorCPU build; full 11-gate record battery greenThis push used
--no-verifybecause the running hook is the pre-fix copy in the primary worktree; all four of its checkers were run manually against the exported tree first.Separately noted in
state.md, not fixed here:check-doc-checkpointis also RED on1a021b1bitself (that policy commit touched 70+.agents/files withoutdocs/BENCHMARKS.md/docs/FEATURES.md); it validates HEAD, so it self-heals as compliant commits land.